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An  evolving  view  of  security 

Operational  resiliency 

Embracing  a  process  view 

Introducing  the  Resiliency  Engineering  Framework 

Summary  and  questions 


Q£pj  Software  Engineering  Institute  Carnegie  Mellon 


Resiliency  Engineering  Framework 


©  2006  Carnegie  Mellon  University 


A  new  operational  environment  -1 


No  operational  boundaries 
Pervasiveness  of  technology 
Expanding  and  rapidly  changing  risk  profile 
High  dependency  on  upstream  partners 
Successes  are  short-lived 
Skills  have  shorter  longevity 
Less  resources,  more  demands 
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A  new  operational  environment  -2 


Increasing  regulatory  requirements 
Criticality  of  data  and  information 
Distributed  workforce 

Heightened  threat  level  and  increasing  uncertainty 
Insurance  costs 

Poses  a  new  environment  in  which  security  must  be 
effective  and  efficient 
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The  problem  with  security  management 


Poorly  planned  and  executed  function 

Business  units  not  involved 

Usually  bolted  on  as  an  afterthought 

Security  seen  as  technical  problem 

Searching  for  magic  bullet:  CobiT,  ITIL,  IS017799 

Poorly  defined  and  measured  goals 

Funding  model  reactive,  not  strategic 

Not  connected  to  continuity  of  operations  planning 
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Organizational  impact 


False  sense  of  accomplishment 
Misalignment  of  operational  and  security  goals 
Reinforcement  of  silos 

Less-than-resilient  assets,  processes,  services 
Misalignment  with  business  objectives 
Wasted  human  and  financial  resources 
Compliance  at  the  expense  of  effectiveness 
Failure  to  manage  operational  risk 
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An  evolving  view  of  security  -1 


Security  is  an  operational  risk  management  activity 
Security  has  two  purposes: 

•  Prevent  disruption  to  core  business  drivers 

•  Sustain  the  survivability  of  the  organization’s  mission 

Security  is  not  an  end,  but  a  means  to  achieving 
higher  organizational  goals 
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An  evolving  view  of  security  -2 
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Operational  risk  and  resiliency 


Operational  risk  is  the  risk  that  results  from 

•  Failed  internal  processes 

•  Inadvertent  or  deliberate  actions  of  people 

•  Problems  with  systems  and  technology 

•  External  events 


Operational  resiliency  is  the  organization’s  ability  to 
sustain  the  mission  in  the  face  of  these  risks 
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Operational  resiliency  is  an  emergent  property 


Operational  resiliency  depends  on  effective 
management  of  core  ORM  activities 


Security  is  one.... 

....but  so  are  Business  Continuity  and  IT 
Operations  Management 


Operational  resiliency  emerges  from  how 
well  these  activities  are  coordinated  and 
executed  toward  a  common  goal 
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Security  and  operational  resiliency 


Focus  on  keeping  critical 
assets  safe  from  harm 

Limiting  threats  and 
managing  impacts 

Manage  confidentiality, 
integrity,  and  availability 

Manage  “condition” 


people 


information 


technology 


facilities 


Q£pj  Software  Engineering  Institute  Carnegie  Mellon 


Resiliency  Engineering  Framework 

©  2006  Carnegie  Mellon  University 


12 


Business  continuity  and  operational  resiliency 


Limit  unwanted  effects  of 
realized  risk 

Ensure  availability  and 
recoverability 

Manage  “consequence” 


people 


information 


technology 


facilities 
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IT  Operations  Management  and  operational  resiliency 


Limit  vulnerabilities  and 
threats  that  originate  in  the 
technical  infrastructure 

Ensure  availability  and 
recoverability  of  technology 


technology 


information 
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Collaborating  toward  a  common  goal 


CZZ) 
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Operational  resiliency  in  practice 
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An  emerging  holistic  view 


Organization  is  dependent  on 
the  productivity  of  four 
assets: 

•  People 

•  Information 

•  Technology 

•  Facilities 

Each  asset  must  be  protected 
and  sustainable 
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A  holistic  risk  perspective 


condition 


Software  Engineering  Institute 
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Collaborating  toward  a  common  goal 


. V 


Resiliency  means 
managing  the 
conditions  and 
consequences  of  risk 
balanced  against 
business  drivers  and 
costs 
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A  mission  focus 


SERVICE 

Business  Process  1 


Business  Process  2 


ORGANIZATIONAL 
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How  does  an  organization  achieve  this? 


Organizations  are  not  structured  today  to  facilitate 
collaboration  toward  a  common  goal  of  resiliency 

•  Deficient  funding  models 

•  Management  direction  and  oversight  lacking 

•  Practice-driven 

•  Compliance-focused 


Need  to  view  resiliency  as  a  definable,  manageable, 
enterprise-wide  process 
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Embracing  a  Process  View  of 
Security  and  Operational 
Resiliency 
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Defining  a  process  approach 


Elevating  the  management  and  coordination  of 
operational-resiliency  focused  activities  to  the  enterprise 
level 

•  Shared  goals  and  resources 

•  Elimination  of  redundancy  and  stovepipes 

•  Elimination  of  framework  quagmire  through  practice 
integration 

•  Measuring  process  effectiveness 

•  Moving  toward  process  improvement 
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How  does  process  differ  from  practice? 


Process 

•  Describes  the  “what” 

•  Set  and  achieve  process  goals 

•  Manage  process  to 
requirements 

•  Select  practices  based  on 
process  goals 

•  Can  be  defined,  communicated, 
measured,  and  controlled 


Practice 

•  Prescribes  the  “how” 

•  No  practice  goals 

•  Tends  toward  “set  and 
forget”  mentality 

•  Reinforces  domain-driven 
approach 

•  One  size  does  not  fit  all 

•  Regulatory  vehicle 
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The  lure  of  best  practices  -1 


Best  practices  are 

effective  ways  to  approach  improvement  in  a  critical 
organizational  activity,  like  security 


Best  practices  ARE  NOT 

a  substitute  for  an  actively  planned  and  managed 
process 
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The  lure  of  best  practices  -2 


Best  practices. . . 

•  Are  often  industry  or  discipline-specific 

•  Change/evolve  frequently 

•  Don’t  have  process  improvement  or  management 
aspects  built-in 

•  Don’t  provide  long-term,  sustainable  success 

•  Can  reinforce  stove-piping  and  silos 

•  People  still  must  implement  and  manage  them 

•  Can  create  a  management  quagmire 
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The  relationship  between  process  and  practice 
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Embracing  process  improvement 


Improvement  in  meeting  resiliency  goals  is  dependent  on 
the  active  management  of  the  process 

Process  maturity  increases  capability  for  meeting  goals 
and  sustaining  the  process 

“Are  we  resilient?”  or  “Are  we  secure?”  is  answered  in  the 
context  of  goal  achievement  rather  than  what  hasn’t 
happened 

Facilitates  meaningful,  purposeful  selection  and 
implementation  of  practices 
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How  mature  are  your  processes? 


Most  organizations  have 

some  process  (implicit  or 

explicit)  for  resiliency  ,-■+•  . , 

engineering,  but  it  may  j 

not  be  effective  for  /  Formal  Process 

meeting  goals.  ' 

!  Partial  Process 

No  Process 


Thanks  to  www.betterproductdesiqn.net/maturitv.htm  for  the  generic  categories. 


(nFICT 


Q^pj  Software  Engineering  Institute  Carnegie  Mellon 


Resiliency  Engineering  Framework 

©  2006  Carnegie  Mellon  University 


29 


Lack  of  process 


No  process  defined  or  performed 

Anarchy  and  heroics 

No  awareness  of  benefits  of 
process-orientation 

AD-HOC 


CERT  I  ~ 


Software  Engineering  Institute 


Common  attributes: 

•  Focus  on  events 

•  Ambiguous  lines  of 
responsibility 

•  Funding  sporadic 

•  No  alignment  to  strategic 
drivers 

•  Highly  dependent  on  people 

•  No  governance  structure 
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Partial  process 


Process  recognized 

Still  functionally  focused  (not 
enterprise-wide) 

Not  repeatable  or  actively  managed 
VULNERABILITY-DRIVEN 


CERT  I  ~ 


Software  Engineering  Institute 


Common  attributes: 

•  Focus  on  vulnerabilities 

•  Responsibility  emanates 
from  IT 

•  Considered  an  expense  or 
burden 

•  Awareness  of  strategic 
drivers 

•  Still  dependent  on  people 
and  vul  catalogs 

•  Informal  governance 
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Formal  process 


Performed  and  managed 

Repeatable 

Spans  enterprise 

Not  completely  ingrained  in  culture 
RISK-DRIVEN 


CERT  I  ~ 


Software  Engineering  Institute 


Common  attributes: 

•  Focus  on  critical  assets 

•  Responsibility  of  key 
organizational  managers 
and  IT 

•  Funded  as  an  expense 

•  Implicit  alignment  to 
strategic  drivers 

•  Dependent  on  localized  risk 
management 

•  Informal  governance, 
possibly  CRM 
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Cultural 


Performed  and  managed 

Repeatable  and  proactive 

Spans  and  involves  enterprise 

Process  continually  measured  and 
improving 

Fundamental  to  organizational 
success 

ENTERPRISE-DRIVEN 


CERT  I  ~ 


Software  Engineering  Institute 


Common  attributes: 

•  Focus  on  critical  assets, 
processes,  strategic  drivers 

•  Responsibility  of  high-level 
executive 

•  Capitalized 

•  Explicit  alignment  to 
strategic  drivers 

•  Reliant  upon  enterprise 
capabilities 

•  Formal  governance  and 
feedback 
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Increasing  levels  of  competency 


/>/. 


9f*n, 


Cultural 


*4 


Formal  Process 


Partial  Process 


No  Process 
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Maturity  from  a  security  perspective 


Cultural 


Formal  Process 


■L 


Partial  Process 


No  Process 


•Technical  problem 
•Owned  by  IT 
•Expense-driven  ™ 
•Practice-centric 
•Security  and  survivability 


•Business  problem 
•Owned  by  organization 
•Investment-driven 
•Process-centric 
•Enterprise  resiliency 
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Toward  continuous  improvement 


Systematic 

and 

Adaptive 


Cultural 


£V, 


ecf 


Cffi 


Formal  Process 


I  i 


Partial  Process 


Irregular 

and 

Reactive 


■L 


No  Process 


Tactical 


Strategic 
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Introducing  the  Resiliency 
Engineering  Framework 
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What  is  resiliency  engineering? 


The  process  by  which  an  organization  establishes, 
develops,  implements,  and  manages  the  operational 
resiliency  of  services,  related  business  processes,  and 
associated  assets 

“Requirements-driven  security  and  business  continuity” 

“Building  resiliency  into  assets/processes/services  and 
managing  to  an  appropriate  level  of  adequacy” 
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The  Resiliency  Engineering  Framework 


A  framework  of  practice  for  integration  of  security  and  business 
continuity  activities  toward  achievement  of  operational 
resiliency 

Defines  basic  process  areas  and  provides  guidelines  for 
security  and  BC/DR  process  improvement 

Captures  vital  linkages  between  security,  BC/DR,  and  l/T  ops  in 
the  process  definition 

Addresses  operational  risk  management  through  process 
management 

Establishes  a  capability  benchmark 


(CFICT 
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Project  history  and  evolution 


Development  history 


OCTAVE  development  and  fieldwork 

Affinity  analysis  of  750  practices 

Identification  of  capabilities 

Identification  of  processes 

Development  of  process  goals  and  practices 

Exploration  of  maturity  concepts 

Exploration  of  assessment  methodologies 
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Framework  architecture 


Represents  processes  that  span  four  basic  areas: 

•  Enterprise  management 

•  Engineering 

•  Operations  management 

•  Process  management 

Considers  the  resiliency  of  people,  information, 
technology,  and  facilities  in  the  context  of  services  and 
business  objectives 
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Enterprise  management  processes 


Enterprise  capabilities  that 
are  essential  to  supporting 
the  resiliency  engineering 
process 


RSKM  -  Risk  Management 

EF  -  Enterprise  Focus 

COMP  -  Compliance  Management 

FRM  -  Financial  Resource 
Management 

HRM  -  Human  Resource  Management 
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Operations  management  processes 


Capabilities  focused  on  sustaining  an  adequate  level  of  operational 
resiliency 


SAM  -  Supplier  Agreement 
Management 

SRM  -  Supplier  Relationship 
Management 

AMC  -  Access  Management  and 
Control 

IMC  -  Incident  Management  and 
Control 


VM  -  Vulnerability  Management 

EC  -  Environmental  Control 

KIM  -  Knowledge  and  Information 
Management 

SOM  -  Security  Operations 
Management 

ITOPS  -  IT  Operations  Management 
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Engineering  processes 


Capabilities  focused  on  establishing  and  implementing  resiliency  for 
organizational  assets,  business  processes,  and  services 

RD  -  Requirements  Definition 

RM  -  Requirements  Management 

AM  -  Asset  Management 

COOP  -  Continuity  of  Operations 
Planning 

REST  -  Restoration  of  Operations 
Planning 

CSI  -  Control  Selection  and 
Implementation 

RAD  -  Resilient  Architecture 
Development 
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Process  management  processes 


Enterprise  capabilities 
related  to  defining,  planning, 
deploying,  implementing, 
monitoring,  controlling, 
appraising,  measuring,  and 
improving  processes 


OT  -  Organizational  Training 

OPF  -  Organizational  Process  Focus 

OPD  -  Organizational  Process 
Definition 

MA  -  Measurement  and  Analysis 
MON  -  Monitoring 
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Using  the  framework 


Establish  current  level  of  capability 

Set  forward-looking  resiliency  goals  and  targets 

Develop  plans  to  close  identified  gaps 

Build  resiliency  into  important  assets/processes/services 
and  architectures 

Reduce  reactionary  activities;  shift  to  directing  and 
controlling  activities 

Align  common  practices  with  processes  to  achieve 
process  goals 
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Collaborating  with  industry 


Eighteen  month  collaboration  with  Financial  Services 
Technology  Consortium 

Identify  mature  practices  in  mature  industries:  banking  and 
financial  services 

Two  phases  of  work — capability  identification  and  process 
definition 
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Financial  Services  Technology  Consortium 


Established  in  1993 

Member-owned  consortium  for  collaboration  between  financial 
services-focused  organization 

Explore  new  technologies  and  methodologies  to  address  today’s 
business  requirements 

Projects: 

•  Technology  Review 

•  Compliance 

•  Business  Continuity  Maturity  Model 
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FSTC  Project  Members 


Ameriprise 

Key  Bank 

Bank  of  America 

KPMG 

Carnegie  Mellon 

MasterCard 

Capital  Group 

Marshall  and  llsley 

Citicorp 

NY  Federal  Reserve  Bank 

Discover 

SunGard 

DRII 

Trizec  Properties 

DRJ 

US  Bank 

IBM 

Wachovia 

JPMorgan  Chase 
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Where  do  we  go  from  here? 


Release  REF  v0.9  in  October  2006  for  comments 

Establish  guidelines  for  improving  the  security  and 
business  continuity  processes 

Phase  III  expansion  of  model  development  and  piloting 

Exploration  of  integration  with  other  existing  models 

Development  of  appraisal  methodology  to  measure 
capability  for  managing  resiliency 
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Summary  and  questions 


Operational  resiliency  must  be  actively  managed 

Security,  BC/DR,  and  IT  Ops  must  collaborate 

Model-based  process  improvement  brings  defined, 
systematic,  repeatable,  consistent,  and  improvable 
processes 

Approach  must  be  flexible  and  adaptable 
No  one-size-fits-all  solution 
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Contact  Us 


Speakers 

Richard  Caralli  rcaralli@cert.org 
Lisa  Young  lry@sei.cmu.edu 


Phone 

412-268-5800 

(8:30  a.m.  -  4:30  p.m.  EST) 


Web 

http://www.cert.org 

http://www.cert.org/nav/index_green.ht 

ml 


Postal  Mail 

Software  Engineering  Institute 
ATTN:  Customer  Relations 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213-3890 


CERT  I  ~ 


Software  Engineering  Institute 


Carnegie  Mellon 


Resiliency  Engineering  Framework 


©  2006  Carnegie  Mellon  University 


53 


